Camera mobile device stabilization using a high frame rate auxiliary imaging element

ABSTRACT

A method to stabilize image capture for a camera device (110). The method includes capturing, at an auxiliary frame rate and using an auxiliary imaging element (112) rigidly coupled with the camera device (110), auxiliary images of a light source (143) attached to an object (142), where the auxiliary frame rate exceeds a camera frame rate of the image capture (1211); detecting each of a plurality of locations of the light source (143) in a corresponding one of the auxiliary images (1212); analyzing the plurality of locations to generate a measure of uncontrolled movement (1213); and adjusting, based at least on the measure of uncontrolled movement, the camera device (110) during the image capture of the object by the camera device (110) at the camera frame rate (1214).

BACKGROUND

A field-of-view (FOV) is an extent of a scene that is imaged by acamera. An object inside the FOV will appear in an image captured and/oroutputted by the camera. For example, the FOV may correspond to a solidangle within which a camera lens projects light input to an opticalsensor of the camera. During image capturing using a handheld camera,movement of the camera often causes unintentional changes of objectposition in the captured images.

Hand shaking of a camera user often results in blurry or out-of-focusimages or videos. Gimbal or stabilizer are used for stabilizing a cameradevice during image capturing. Such solutions require two or more motionsensors, e.g., gyroscopes, with associated electronic circuits to obtaininformation for computing control signal(s) to correct the user handshaking. Motion sensors and associated electronic circuits increasemanufacturing cost and product size.

SUMMARY

In general, in one aspect, the invention relates to a method tostabilize image capture for a camera device. The method includescapturing, at an auxiliary frame rate and using an auxiliary imagingelement rigidly coupled with the camera device, a plurality of auxiliaryimages of a light source attached to an object, wherein the auxiliaryframe rate exceeds a camera frame rate of the image capture, detectingeach of a plurality of locations of the light source in a correspondingone of the plurality of auxiliary images, analyzing the plurality oflocations to generate a measure of uncontrolled movement, and adjusting,based at least on the measure of uncontrolled movement, the cameradevice during the image capture of the object by the camera device atthe camera frame rate.

In general, in one aspect, the invention relates to a stabilized cameradevice holder for stabilizing image capture. The stabilized cameradevice holder includes a computer processor, memory storinginstructions, when executed, causing the computer processor to receive,from an auxiliary imaging element rigidly coupled with a camera device,a plurality of auxiliary images of a light source attached to an object,wherein the plurality of auxiliary images are captured by the auxiliaryimaging element at an auxiliary frame rate that exceeds a camera framerate of the image capture, detect each of a plurality of locations ofthe light source in a corresponding one of the plurality of auxiliaryimages, analyze the plurality of locations to generate a measure ofuncontrolled movement, and adjust, based at least on the measure ofuncontrolled movement, the camera device during the image capture of theobject by the camera device at the camera frame rate, and at least onemotor configured to adjust, based at least on the measure ofuncontrolled movement, the camera device during the image capture of theobject by the camera device at the camera frame rate.

In general, in one aspect, the invention relates to a stabilizationcontroller device for stabilizing image capture. The stabilizationcontroller device includes a computer processor and memory storinginstructions, when executed, causing the computer processor to receive,from an auxiliary imaging element rigidly coupled with a camera device,a plurality of auxiliary images of a light source attached to an object,wherein the plurality of auxiliary images are captured by the auxiliaryimaging element at an auxiliary frame rate that exceeds a camera framerate of the image capture, detect each of a plurality of locations ofthe light source in a corresponding one of the plurality of auxiliaryimages, analyze the plurality of locations to generate a measure ofuncontrolled movement, and adjust, based at least on the measure ofuncontrolled movement, the camera device during the image capture of theobject by the camera device at the camera frame rate.

In general, in one aspect, the invention relates to a non-transitorycomputer readable medium storing instructions for stabilizing imagecapture for a camera device. The instructions, when executed by acomputer processor, comprising functionality for capturing, at anauxiliary frame rate and using an auxiliary imaging element rigidlycoupled with the camera device, a plurality of auxiliary images of alight source attached to an object, wherein the auxiliary frame rateexceeds a camera frame rate of the image capture, detecting each of aplurality of locations of the light source in a corresponding one of theplurality of auxiliary images, analyzing the plurality of locations togenerate a measure of uncontrolled movement, and adjusting, based atleast on the measure of uncontrolled movement, the camera device duringthe image capture of the object by the camera device at the camera framerate.

Other aspects of the invention will be apparent from the followingdescription and the appended claims.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1.1 and 1.2 show a schematic block diagram of a system inaccordance with one or more embodiments of the invention.

FIGS. 1.3 and 1.4 show example diagrams in accordance with one or moreembodiments of the invention.

FIGS. 2.1 and 2.2 show method flowcharts in accordance with one or moreembodiments of the invention.

FIGS. 3.1, 3.2, 3.3, 3.4, 3.5, 4, 5, and 6 show various examples inaccordance with one or more embodiments of the invention.

FIGS. 7.1 and 7.2 show a computing system in accordance with one or moreembodiments of the invention.

DETAILED DESCRIPTION

Specific embodiments of the invention will now be described in detailwith reference to the accompanying figures. Like elements in the variousfigures may be denoted by like reference numerals for consistency.

In the following detailed description of embodiments of the invention,numerous specific details are set forth in order to provide a morethorough understanding of the invention. However, it will be apparent toone of ordinary skill in the art that the invention may be practicedwithout these specific details. In other instances, well-known featureshave not been described in detail to avoid unnecessarily complicatingthe description.

In the following description, any component described with regard to afigure, in various embodiments of the invention, may be equivalent toone or more like-named components described with regard to any otherfigure. For brevity, at least a portion of these components areimplicitly identified based on various legends. Further, descriptions ofthese components will not be repeated with regard to each figure. Thus,each and every embodiment of the components of each figure isincorporated by reference and assumed to be optionally present withinevery other figure having one or more like-named components.Additionally, in accordance with various embodiments of the invention,any description of the components of a figure is to be interpreted as anoptional embodiment which may be implemented in addition to, inconjunction with, or in place of the embodiments described with regardto a corresponding like-named component in any other figure. In thefigures, black solid collinear dots indicate that additional componentssimilar to the components before and/or after the solid collinear dotsmay optionally exist. Further, a solid line or a dash line connectingthe components of a figure represent a relationship between theconnected components. The dash line indicates that the relationship maynot include or otherwise associate with any physical connection orphysical element.

Throughout the application, ordinal numbers (e.g., first, second, third,etc.) may be used as an adjective for an element (i.e., any noun in theapplication). The use of ordinal numbers is not to imply or create anyparticular ordering of the elements nor to limit any element to beingonly a single element unless expressly disclosed, such as by the use ofthe terms “before”, “after”, “single”, and other such terminology.Rather, the use of ordinal numbers is to distinguish between theelements. By way of an example, a first element is distinct from asecond element, and the first element may encompass more than oneelement and succeed (or precede) the second element in an ordering ofelements.

In general, embodiments of the invention provide a method, a device, asystem, and a computer readable medium for stabilizing a camera deviceduring image capturing. In particular, auxiliary images of a lightsource attached to an object are captured using an auxiliary imagingelement. The auxiliary imaging element is rigidly coupled with thecamera device and has an auxiliary frame rate that exceeds the cameraframe rate. Locations of the light source in the auxiliary images aredetected and analyzed to generate a measure of uncontrolled movement.The measure of uncontrolled movement is used to adjust the camera deviceduring the image capture of the object by the camera device. In one ormore embodiments, adjusting the camera device is by generating a controlsignal based on the measure of uncontrolled movement to adjust astabilized camera device holder holding the camera device. Inparticular, the measure of uncontrolled movement may represent user handshaking when holding the stabilized camera device holder. Suchadjustment mitigates image blurring of the camera device caused by theuser hand shaking.

In one or more embodiments, the locations of the light source detectedin the auxiliary images are further analyzed to generate a measure ofcontrolled movement. In one or more embodiments, the control signal isgenerated further based on the measure of controlled movement to adjustthe stabilized camera device holder. In particular, the measure ofcontrolled movement may represent movement of the object in the scene.Such adjustment allows the stabilized camera device holder to track theobject movement such that the object appears at a target position in theimages generated by the camera device. In one or more embodiments, themeasure of controlled movement is used to mitigate a rolling shuttereffect of the camera device.

FIG. 1.1 shows a system (100) in accordance with one or moreembodiments. In one or more embodiments, one or more of the modules andelements shown in FIG. 1.1 may be omitted, repeated, and/or substituted.Accordingly, embodiments of the invention should not be consideredlimited to the specific arrangements of modules shown in FIG. 1.1.

As shown in FIG. 1.1, the system (100) includes a camera device (110)having an imaging element A (111), an stabilization controller (120)associated with an imaging element B (112), a camera device holder(130), a scene (140), a field-of-view (FOV) (141), an object (142)appearing within the FOV (141), and a light source (143) attached to theobject (142). Further, the camera device (110) with the imaging elementA (111), the stabilization controller (120) with the imaging element B(112), and the camera device holder (130) are communicatively coupled toeach other. In particular, the stabilization controller (120) and thecamera device holder (130) are collectively referred to as a stabilizedcamera device holder (135). In one or more embodiments of the invention,two or more of the camera device (110), the stabilization controller(120), the imaging element B (112), and the camera device holder (130)are integrated into a single device. For example, at least a portion ofthe stabilization controller (120) may be included in the camera device(110). In another example, at least a portion of the stabilizationcontroller (120) may be included in the camera device holder (130). Instill another example, one part of the stabilization controller (120) isincluded in the camera device (110) while another part of thestabilization controller (120) is included in the camera device holder(130). In a further example, the imaging element B (112) may beintegrated with the stabilization controller (120) or the camera deviceholder (130). Depending on which of the examples, the stabilized cameradevice holder (135) may be a logical combination or a physicalcombination of the stabilization controller (120) and the camera deviceholder (130). In one or more embodiments, the imaging element A (111)and the imaging element B (112) are rigidly coupled to each other suchthat any user hand shaking affects both the imaging element A (111) andthe imaging element B (112) to the same extent. For example, the imagingelement B (112) may be embedded in the camera device holder (130), whichrigidly holds the camera device (110) where the imaging element A (111)is a part of.

In one or more embodiments of the invention, the imaging element A (111)of the camera device (110) is configured to generate (i.e., capture)images (e.g., image X (126 x) depicted in FIG. 6 below) of the scene(140). In one or more embodiments, the imaging element A (111) issensitive to visible light (e.g., having wavelength in the range of 390to 700 nanometers (nm)) and the captured images (e.g., image X (126 x)depicted in FIG. 6 below) are visible images, such as a photograph orframes in a video recording. In one or more embodiments of theinvention, the imaging element B (112) is configured to generateauxiliary images (e.g., image A (126 a) depicted in FIGS. 1.2, 5, and 6below) of the scene (140). In this context, the imaging element B (112)is referred to as an auxiliary imaging element. In particular, theimaging element B (112) is sensitive to the light emitted or reflectedby the light source (143) such that the light source (143) appears inthe captured auxiliary images. In one or more embodiments, the lightsource (143) emits or reflects infrared light (e.g., having wavelengthin the range of 700 nm to 1 millimeter (mm)) and the imaging element B(112) is an infrared imaging element. As described below, the auxiliaryimages are used to mitigate user hand shaking and stabilize imagecapture of the camera device (110). In this context, the imaging elementB (112) is referred to as an auxiliary imaging element. For example, theimaging element A (111) and imaging element B (112) may include a CMOS(complementary metal oxide semiconductor) cell array, a CCD (chargecoupled device) array, or other types of photo-sensor array that aresensitive to visible light and/or infrared light.

In one or more embodiments of the invention, the light source (143) isany device that emits or reflects light, such as infrared lightdescribed above. In one or more embodiments, the light source (143)includes a light-emitting-diode (LED). In one or more embodiments, thelight source (143) includes a reflective region of (or attached to) theobject (142). In one or more embodiments, the light source (143) emits astrobe light, which changes intensity and/or color from time to time. Inone or more embodiments, the light source (143) reflects the strobelight emitted by a remote light emitter (not shown). For example, thestrobe light may emit a free-running light change pattern according to aparticular duty cycle (i.e., a percentage of time when the light patternhas a bright level) and repetition rate (i.e., a number of time theintensity changes during a unit time period). As used herein, lightchange pattern is a pattern of intensity and/or color change in thelight. In one or more embodiments, the light source (143) emits orreflects a light change pattern with a high repetition rate (e.g., 1kilohertz (kHz), 2 kHz, etc.) comparing to a camera frame rate of thecamera device (110) and an auxiliary frame rate of the imaging element B(112). The camera frame rate is a number of images (e.g., a burst ofstill images or a video recording) captured by the imaging element A(111) of the camera device (110) during a unit time. The auxiliary framerate is the number of images (e.g., a burst of still images or a videorecording) captured by the imaging element B (112) during the unit time.In one or more embodiments, the light source (143) emits or reflects alight change pattern that is synchronized with the camera frame rate ofthe imaging element A (111) and/or the auxiliary frame rate of theimaging element B (112). In one or more embodiments, the camera framerate of the imaging element A (111) is 24 Hz, 25 Hz, 30 Hz, 50 Hz, or 60Hz. In one or more embodiments, the auxiliary frame rate of the imagingelement B (112) is between 6 times and 10 times of the camera framerate.

In one or more embodiments of the invention, the camera device (110) isa device with the imaging element A (111) and associated components fortaking photographs and/or video recordings. A dedicated camera (e.g., apoint-and-shoot camera or a digital-single-lens-reflective (DSLR)camera) with communication capability (e.g., Bluetooth, WiFi, etc.) isan example of the camera device (110). In one or more embodiments, thecamera device (110) is a mobile device, such as a mobile phone with abuilt-in camera, referred to as a smart phone. A smart phone may have adisplay with graphical user interface that occupy a large portion (e.g.,70% or larger) of the front surface. The imaging element A (111) may beon the front surface or back surface of the smart phone. In one or moreembodiments, the camera device (110) includes a hardware component, asoftware component, or a combination thereof. In one or moreembodiments, the camera device (110) may include, or otherwise beimplemented using, at least a portion of the computing system (700) andnetwork (720) described in reference to FIGS. 7.1 and 7.2 below.

In one or more embodiments, the scene (140) is a place where an actionor event, imaged by the camera device (110), occurs. The field-of-view(FOV) (141) corresponds to an extent of the scene (140) that is imagedby the camera device (110) using the imaging element A (111), thus isreferred to the FOV of the camera device (110) or FOV of the imagingelement A (111). The object (142) inside the FOV (141) will appear in animage captured and/or outputted by the camera device (110). The FOV(141) may also correspond to an extent of the scene (140) that is imagedby the imaging element B (112), thus is referred to as the FOV of theimaging element B (112). The light source (143) inside the FOV (141)will appear in an auxiliary image captured and/or outputted by theimaging element B (112). For example, the FOV (141) may correspond to asolid angle within which the imaging element A (111) or imaging elementB (112) receives light input to an associated optical sensor (not shown)of the imaging element A (111) or imaging element B (112). In one ormore embodiments, the FOV (141) corresponds to different portions of thescene (140) according to how the imaging element A (111) and imagingelement B (112) oriented toward, zoomed with respect to, or otherwisepositioned relative to, the scene (140). Based on the rigid coupling ofthe imaging element A (111) and imaging element B (112), the FOV of theimaging element A (111) may be a rigid portion of the FOV of the imagingelement B (112), or vice versa. In other words, in some cases, the FOVof the imaging element B (112) may be a rigid portion of the FOV of theimaging element A (111). In a particular case, the FOV of the imagingelement A (111) may be substantially aligned with the FOV of the imagingelement B (112).

In one or more embodiments of the invention, the camera device holder(130) is configured to mechanically hold the camera device (110) and toadjust, in response to a control signal from the stabilizationcontroller (120), the FOV (141) of the imaging element A (111). Forexample, the camera device holder (130) may include a motorized tilt andswivel device for adjusting a camera angle of the imaging element A(111). In another example, the camera device holder (130) may include amotorized horizontal and vertical sliding device for adjusting aposition of the imaging element A (111) relative to the scene (140). Thesliding device may include a mechanical stage for holding and moving thecamera device (110). Examples of the camera device holder (130) aredescribed in reference to FIGS. 3.1-3.5 below.

In one or more embodiments, the stabilization controller (120) includesa hardware component, a software component, or a combination thereofthat is configured to generate the aforementioned control signal toadjust the FOV (141) of the imaging element A (111). In particular, thestabilization controller (120) uses a pre-determined algorithm togenerate the control signal based on the auxiliary images captured bythe imaging element B (112). In one or more embodiments, thestabilization controller (120) includes a timer to control the auxiliaryframe rate of the imaging element B (112) based on the duty cycle and/orrepetition rate of the light source (143). An example of generating thecontrol signal based on the auxiliary images is described in referenceto FIGS. 4-6 below. For example, the stabilization controller (120) maycontrol the FOV (141) by way of controlling the camera device holder(130) using the control signal. In another example, the stabilizationcontroller (120) may further control the FOV (141) by way of controllinga zoom level of the imaging element A (111) using the control signal. Inone or more embodiments, the stabilization controller (120) controls theFOV (141) such that the object (142) appears in a stable position withinthe FOV (141). In one or more embodiments, the stabilization controller(120) controls the FOV (141) such that the object (142) moves toward atarget position within the FOV (141). In one or more embodiments, thestabilization controller (120) controls the FOV (141) such that theobject (142) moves with improved stability/smoothness toward a targetposition within the FOV (141). In one or more embodiments, thestabilization controller (120) controls the FOV (141) using the methoddescribed in reference to FIGS. 2.1 and 2.2 below. In one or moreembodiments, the stabilization controller (120) include the componentsdescribed in reference to FIG. 1.2 below.

FIG. 1.2 shows details of the stabilization controller (120) inaccordance with one or more embodiments. The following description ofFIG. 1.2 refers to various components depicted in FIG. 1.1 above. In oneor more embodiments, one or more of the modules and elements shown inFIG. 1.2 may be omitted, repeated, and/or substituted. Accordingly,embodiments of the invention should not be considered limited to thespecific arrangements of modules shown in FIG. 1.2.

As shown in FIG. 1.2, the stabilization controller (120) includes ahardware processor (121), memory (122), and repository (123). In one ormore embodiments of the invention, the hardware processor (121)corresponds to the computer processor(s) (702) depicted in FIG. 7.1below. Similarly, the memory (122) and repository (123) correspond tothe non-persistent storage (704) and/or persistent storage (706)depicted in FIG. 7.1 below. For example, the memory (122) may storesoftware instructions that, when executed, cause the hardware processor(121) to perform FOV adjustment functionalities of the camera device(110) depicted in FIG. 1.1 above. In one or more embodiments, the FOVadjustment functionalities stabilize image captures of the camera device(110). In one or more embodiments, the FOV adjustment functionalitiesincludes object tracking during image captures of the camera device(110). In one or more embodiments, the stabilization controller (120)performs the FOV adjustment functionalities according to the methodflowchart described in reference to FIGS. 2.1 and 2.2 below. In one ormore embodiments, the memory (122) stores instructions to perform one ormore portions of the method flowchart described in reference to FIGS.2.1 and 2.2 below. In one or more embodiments, at least a portion of thestabilization controller (120) and the camera device (110) areintegrated into a single device. In such embodiments, the instructionsto perform one or more portions of the method flowchart described inreference to FIGS. 2.1 and 2.2 are part of a mobile application, ormobile app, which is a user-installable software application designed torun on a smart phone or other mobile devices. In one or moreembodiments, at least a portion of the stabilization controller (120)and the camera device holder (130) are integrated into a single device.In such embodiments, the instructions to perform one or more portions ofthe method flowchart described in reference to FIGS. 2.1 and 2.2 arepart of a local application (e.g., installed software, embeddedfirmware) designed to run on the camera device holder (130).

Further as shown in FIG. 1.2, the repository (123) includes a sequenceof auxiliary images (126), a light change pattern (124), a displacement(125), an uncontrolled movement parameter (125 a), a controlled movementparameter (125 b), a target position (127), a predicted trajectory(128), and a motor control parameter (129). In particular, the sequenceof auxiliary images (126) includes consecutive images (e.g., image A(126 a)) captured by the imaging element B (112). For example, the imageA (126 a) corresponds to a portion of the scene (140) that is covered bythe FOV of the imaging element B (112) at a particular time point.

The light change pattern (124) is a pattern of light intensity and/orcolor alternating between different intensity levels and/or colorsacross the sequence of auxiliary images (126). In one or moreembodiments, the light change pattern (124) corresponds to a spot ineach image of the sequence of auxiliary images (126). For example, thespot may be defined by a pixel position or a collection of connectedpixel positions in each image. In one or more embodiments, the lightchange pattern (124) is caused by a strobe light emitted from orreflected by the light source (143) and indicates a location of thelight source (143) within each image. In other words, the location ofthe light source (143) within each image may be determined based onwhere the light change pattern (124) is found across the sequence ofauxiliary images (126). For example, the light change pattern (124)indicates that the light source (143) is at the location A (126 b) inthe image A (126 a). Similarly, each other image in the sequence ofauxiliary images (126) is associated with a location of the light source(143).

In one or more embodiments, the light source (143) is attached to theobject (142) such that the location of the light source (143) in animage represents the location of the object (142). The target position(127) is a pre-determined position that the stabilization controller(120) is configured for tracking the object (142) during image captures.For example, the target position (127) may be defined as the center ofthe FOV (141), which corresponds to the center of the image. In otherwords, the stabilization controller (120) is configured to adjust theFOV (141) such that the object (142) appears at the center (i.e., targetposition (127)) in the image after the adjustment. In other examples,the target position (127) may be defined as different positions from thecenter of the FOV (141). In those embodiments where the FOVs of theimaging element A (111) and imaging element B (112) are not aligned witheach other, the target position (127) may be adjusted to account for anyoffset between the center of FOV of the imaging element A (111) and thecenter of FOV of the imaging element B (112).

The displacement (125) is the distance between the target position (127)and the location (e.g., location A (127 a)) of the light source (143)within an image. In one or more embodiments, the displacement (125)includes a horizontal direction distance and a vertical distance. Thedisplacement (125) may be represented based on a number of pixels or anyother suitable distance scale.

The uncontrolled movement parameter (125 a) and controlled movementparameter (125 b) represent variations of the location (e.g., location A(127 a)) of the light source (143) from one image to next in thesequence of auxiliary images (126). As used herein, controlled movementis a predicted trajectory of an object appearing across consecutiveauxiliary images, and uncontrolled movement is a deviation from thepredicted trajectory of the object appearing across consecutiveauxiliary images. In one or more embodiments, the uncontrolled movementparameter (125 a) and the controlled movement parameter (125 b) aremeasured based on image pixels. In one or more embodiments, theuncontrolled movement parameter (125 a) represents user handshaking, andthe controlled movement parameter (125 b) represents movement of amoving object in the scene (140) and/or movement of the camera device(110) relative to the scene (140). In one or more embodiments, theobject (142) is a moving object where the predicted trajectory (128)represents the movement trajectory of the moving object. In one or moreembodiments, the camera device (110) is moving relative to the scene(140) where the predicted trajectory (128) represents the relativemovement between the object (142) from the point of view of the cameradevice (110). For example, the user may be holding the camera deviceholder (130) with the camera device (110) in a traveling vehicle suchthat a stationery object (142) may appear to be moving. Further detailsof the uncontrolled movement parameter (125 a), controlled movementparameter (125 b), and predicted trajectory (128) are described inreference to FIG. 1.3 below.

Because the imaging element A (111) and imaging element B (112) arerigidly coupled to each other, the position where the object (142)appears in the FOVs of the imaging element A (111) or imaging element B(112) depends on the orientation of the camera device (110).Accordingly, the object (142) may be aligned with the target position(127) in the FOV (141) by changing the orientation of the camera device(110). In one or more embodiments, the motor control parameter (129)corresponds to control information for controlling the motors of thecamera device holder (130) to stabilize image capturing or performobject tracking using the camera device (110). In one or moreembodiments, motor control parameter (129) is generated by thestabilization controller (120) based on one or more of the uncontrolledmovement parameter (125 a), controlled movement parameter (125 b), andpredicted trajectory (128) described above. In one or more embodiments,the motor control parameter (129) specifies, for any particular timepoint, a rotation change amount, a tilt change amount, a shift changeamount, or other amount of change to be sent to the camera device holder(130) to mitigate user handshaking or to orient the camera device (110)toward the moving object (142). In one or more embodiments, the motorcontrol parameter (129) is sent to the camera device holder (130) as acontrol signal for controlling the motors thereof. For example, thecontrol signal may be a digital electronic signal or an analogelectronic signal. Examples of controlling the motors of the cameradevice holder (130) are described in reference to FIGS. 3.1-3.5 below.

An example of the sequence of auxiliary images (126), light changepattern (124), target position (127), displacement (125), and motorcontrol parameter (129) is described in reference to FIGS. 4-6 below.

FIG. 1.3 shows example details of the uncontrolled movement parameter(125 a), controlled movement parameter (125 b), and predicted trajectory(128) based on the legend (131). In one or more embodiments, one or moreof the modules and elements shown in FIG. 1.3 may be omitted, repeated,and/or substituted. Accordingly, embodiments of the invention should notbe considered limited to the specific arrangements of modules shown inFIG. 1.3.

As shown in FIG. 1.3, the vertical axis (denoted as X) represents jitter(e.g., horizontal location) of the light source (143) detected in eachof the sequence of auxiliary images (126), and the horizontal axis(denoted as T) represents time. In particular, each circle representsthe vertical location of the light source (143) detected in each evenimage in the sequence of auxiliary images (126), while each solid dotrepresents the vertical location of the light source (143) detected ineach odd image in the sequence of auxiliary images (126). Along thehorizontal axis, Δt represents time between the even and odd images. Inother words, Δt (e.g., 1 ms) equals the inverse of auxiliary frame rateof the imaging element B (112). In addition, Δy (e.g., 5 ms) representsthe sampling time for determining the predicted trajectory (128) of theobject (142). In comparison, the time between image capture of thecamera device (110) using the imaging element A (111) may be 33 ms ifthe camera frame rate is 30 Hz.

In one or more embodiments, the stabilization controller (120) generatesthe predicted trajectory (128) by applying a curve fitting algorithm tothe horizontal locations of the light source (143) detected in thesequence of auxiliary images (126) during each Δy sampling time period.In other words, the predicted trajectory (128) approximates the circlesand solid dots within each Δy sampling time period. In the example shownin FIG. 1.3, the approximated curve is a linear segment within each Δysampling time period. In other example, different type of curvelinearapproximation may be used. Accordingly, the uncontrolled movementparameter (125 a) is computed as deviation of the detected location ofthe light source (143) from the predicted trajectory (128). As describedabove, the uncontrolled movement parameter (125 a) may represents userhandshaking during image captures. Using a stationery reference locationdenoted by the dashed line, the controlled movement parameter (125 b)corresponds to the deviation of the predicted trajectory (128) from thestationary reference location. As described above, the controlledmovement parameter (125 b) may represents relative movement between theobject (142) and the camera device (110).

While FIG. 1.3 depicts horizontal portions of the uncontrolled movementparameter (125 a), controlled movement parameter (125 b), and predictedtrajectory (128), vertical portions of the uncontrolled movementparameter (125 a), controlled movement parameter (125 b), and predictedtrajectory (128) may be determined by the stabilization controller (120)in a similar manner.

FIG. 1.4 shows an example rolling shutter effect in accordance with oneor more embodiments. In one or more embodiments, one or more of themodules and elements shown in FIG. 1.4 may be omitted, repeated, and/orsubstituted. Accordingly, embodiments of the invention should not beconsidered limited to the specific arrangements of modules shown in FIG.1.4.

As shown in FIG. 1.4, the stationery image (140 a) represents a portionof the scene (140) that is captured by the imaging element A (111) orimaging element B (112) when the imaging element A (111) and imagingelement B (112) are stationery with respect to the scene (140). Incontrast, the moving image (140 b) represents a portion of the scene(140) that is captured by the imaging element A (111) or imaging elementB (112) when the imaging element A (111) and imaging element B (112) aremoving relative to the scene (140). For example, the stabilizationcontroller (120) with the camera device (110) may be held by a user in atraveling vehicle. In another example, the camera device (110) may bemoved (e.g., rotated, tilted, etc.) by the stabilization controller(120) to track the object (142) when there is relative movement betweenthe object (142) and the camera device (110). In one or moreembodiments, the imaging element A (111) or imaging element B (112) isbased on a CMOS sensor with a rolling shutter. In other words, thestationery image (140 a) and moving image (140 b) are composed bymultiple scan lines (140 c) that are captured sequentially in time. Therolling shutter effect is the distortion of the moving image (140 b) dueto the relative movement between the object (142) and the camera device(110). As shown in FIG. 1.4, the distortion corresponds to verticalmisalignment (or shift) of the multiple scan lines (140 c) capturedsequentially in time. In one or more embodiments, the predictedtrajectory (128) and/or the controlled movement parameter (125 b) areused to correct the distortion (e.g., misalignment or scan line shift)of the image captured by the camera device (110).

FIG. 2.1 shows a flowchart in accordance with one or more embodiments.In particular, the flowchart is an example of generating theaforementioned control signal for adjusting the FOV of the cameradevice. The process shown in FIG. 2.1 may be executed, for example, byone or more components discussed above in reference to FIGS. 1.1 and1.2. One or more steps shown in FIG. 2.1 may be omitted, repeated,and/or performed in a different order among different embodiments of theinvention. Accordingly, embodiments of the invention should not beconsidered limited to the specific number and arrangement of steps shownin FIG. 2.1.

Initially in Step 1211, a sequence of auxiliary images of an lightsource attached to an object is captured at an auxiliary frame rateusing an auxiliary imaging element. The auxiliary imaging element isrigidly coupled with a camera device.

In Step 1212, each location of a sequence of locations of the lightsource is detected in a corresponding auxiliary image of the sequence ofauxiliary images. In one or more embodiments, the location of the lightsource is detected using the method described in reference to FIG. 2.2below.

In Step 1213, the sequence of locations is analyzed to generate ameasure of controlled movement and a measure of uncontrolled movement.In one or more embodiments, as described in reference to FIG. 1.3 above,the measure of controlled movement and the measure of uncontrolledmovement are generated by applying a curve fitting algorithm to thedetected locations of the light source in the sequence of auxiliaryimages.

In Step 1214, a control signal is generated based on the measure ofuncontrolled movement for adjusting a field-of-view (FOV) of the cameradevice to stabilize the image capture. In one or more embodiments, thecontrol signal is generated to orient the camera device to compensatethe uncontrolled movement.

In Step 1215, the control signal is further generated based on themeasure of controlled movement for adjusting the FOV of the cameradevice to track an object moving in the FOV of the camera device. In oneor more embodiments, the control signal is generated to orient thecamera device to compensate the controlled movement. In one or moreembodiments, the control signal is further generated to track a movingobject using the method described in reference to FIG. 2.2 below.

In Step 1216, distortion in a captured image due to a rolling shuttereffect is corrected based on the measure of controlled movement. In oneor more embodiments, the distorting includes misalignment or shiftbetween scan lines captured by the imaging element of the camera device.Accordingly, the misalignment or shift is compensated based on themeasure of controlled movement. For example, the misaligned scan linesmay be shifted in reverse direction where the extent and direction ofthe reverse shift is based on the measure of controlled movement.

In Step 1217, a determination is made as to whether the image capturinghas completed. If the determination is negative, i.e., the imagecapturing is continuing, the method returns to Step 1211. If thedetermination is positive, i.e., the image capturing is completed, themethod ends.

FIG. 2.2 shows a flowchart in accordance with one or more embodiments.In particular, the flowchart is an example of generating theaforementioned control signal for tracking a moving object. The processshown in FIG. 2.2 may be executed, for example, by one or morecomponents discussed above in reference to FIGS. 1.1 and 1.2. One ormore steps shown in FIG. 2.2 may be omitted, repeated, and/or performedin a different order among different embodiments of the invention.Accordingly, embodiments of the invention should not be consideredlimited to the specific number and arrangement of steps shown in FIG.2.2.

Initially, in Step 1221, a light source within a scene is activated. Inone or more embodiments of the invention, the light source is attachedto an object in the scene. In one or more embodiments, the light sourceemits or reflects a strobe light, which changes intensity and/or colorfrom time to time. For example, the strobe light emits a free-runninglight pattern in response to the light source being activated (e.g.,turned on).

In Step 1222, a sequence of auxiliary images of the scene is captured byan auxiliary imaging element that is rigidly coupled to a camera device.In particular, the object is within the field-of-view (FOV) of theauxiliary imaging element and appears in the sequence of auxiliaryimages. For example, the sequence of auxiliary images may include or bepart of a burst of still images. In another example, the sequence ofauxiliary images may include or be part of a video recording. In one ormore embodiments, the sequence of auxiliary images of the scene iscaptured while the light source emits or reflects the strobe light. Inone or more embodiments, the frame rate of the sequence of auxiliaryimages is selected based on the duty cycle and/or repetition rate of thelight source such that consecutive images (or a pair of images with aparticular separation in the sequence) include alternating bright leveland dark level, and/or alternating colors from the light source. Forexample, the light source may be free running and the frame rate isselected based on a pre-determined duty cycle and/or repetition rate ofthe free running light source. In one or more embodiments, a timer isused to control image capture according to the selected frame rate.

In one or more embodiments, the duty cycle and/or repetition rate of thelight source is selected based on the frame rate of the sequence ofauxiliary images such that consecutive images (or a pair of images witha particular separation in the sequence) include alternating brightlevel and dark level, and/or alternating colors from the light source.For example, the frame rate may be pre-determined and the light sourceis synchronized to the frame rate, e.g., based on a trigger signal fromthe auxiliary imaging element or a stabilization controller thatcontrols the auxiliary imaging element.

In Step 1223, based on a local light change pattern across the sequenceof auxiliary images, the light source is detected in the scene.Specifically, the strobe light emitted from or reflected by the lightsource causes changes in light intensity and/or color received by theauxiliary imaging element resulting in the local light change patternacross the sequence of auxiliary images. In one or more embodiments, theintensity of the strobe light is adjusted to control the size of thelocation where the local intensity change pattern is found in eachimage. For example, the location size may be limited to a percentage(e.g., 1%, 3%, etc.) of the horizontal and vertical dimensions of theFOV. In one or more embodiments, the location and the size are definedwhere the difference in alternating bright level and dark level, and/oralternating colors, in consecutive images, as recognized by theauxiliary imaging element, exceeds a pre-determined threshold. In one ormore embodiments, the location is referred to as the location of thelight source in the image.

In one or more embodiments, a pair of images in the sequence ofauxiliary images are compared by subtraction of intensity and/or colorvalues of corresponding pixels. Specifically, the intensity and/or colorvalues are generated by the auxiliary imaging element. In particular,the intensity and/or color value of a pixel in one image is subtractedfrom the intensity and/or color value of the corresponding pixel inanother image to generate a subtraction result. The pixel where thedifference in alternating bright level and dark level, and/oralternating colors, is found in the subtraction result is selected aspart of the location of the light source in the image. Depending on theduty cycle/repetition rate of the light source versus the frame rate ofthe sequence of auxiliary images, the pair of images may be consecutiveimages or two images separated by a particular number of images, such asevery three images, etc.

In Step 1224, the sequence of auxiliary images is analyzed to determinea location of the light source in at least one image and a movement ofthe light source across the sequence of auxiliary images. In one or moreembodiments, the location of the light source is determined based onwhere the difference in alternating bright level and dark level, and/oralternating colors in the sequence of auxiliary images, as recognized bythe auxiliary imaging element, exceeds the pre-determined threshold. Inone or more embodiments, the movement of the light source is determinedbased on a rate of change of the location over the sequence of auxiliaryimages.

In Step 1225, in response to detecting the light source, the location ofthe light source and a target position within at least one image arecompared to generate a result. In one or more embodiments, the resultincludes the displacement from the location to the target position. Inone or more embodiments, the displacement may vary from one image tonext in the sequence of auxiliary images, indicating that the object isa moving object. In such embodiments, the rate of change of thedisplacement over time, e.g., from one image to next, is computed as amovement parameter.

In Step 1226, a control signal is generated based on the result fororienting the camera device. In one or more embodiments, the controlsignal is configured to adjust the orientation of the camera device inthe opposite direction to the displacement. For example, if thedisplacement indicates that the target position is to the right of thelight source location within the image, the control signal adjusts theorientation of the camera device toward the left. In one or moreembodiments, the control signal is configured to adjust the relativeposition of the camera with respect to the scene in the oppositedirection to the displacement. For example, if the displacementindicates that the target position is to the right of the light sourcelocation within the image, the control signal adjusts the relativeposition of the camera toward the left. In one or more embodiments, themovement parameter is considered in fine tuning the amount of adjustmentcaused by the control signal.

In Step 1227, the control signal is sent to a camera device holder(e.g., a tilt-and-swivel device or a mechanical stage) where the cameradevice is mounted. Accordingly, the orientation of the camera device ora relative position of the camera device is adjusted in the oppositedirection to the displacement. In one or more embodiments, the controlsignal is sent to the camera device holder using a communication linkbetween the camera device and the camera device holder. In one or moreembodiments, the control signal includes a rotating control signal and atilting control signal for controlling one or more rotation motors and atilting motor, respectively of the camera device holder. Using thecontrol signal, a geometrical behavior of the camera device holder isadjusted. In one or more embodiments, the geometrical behaviorcorresponds to rotating, tilting, sliding, or other motion of one ormore components of the camera device holder. In one or more embodiments,adjusting the geometrical behavior of the camera device holder includesactivating, using the rotating control signal, one or more rotatingmotors of the camera device holder, and activating, using the tiltingcontrol signal, a tilting motor of the camera device holder. In one ormore embodiments where a previous orientation of the camera device isused as the target orientation, the target orientation is updated usinga current orientation of the camera device after adjustment using thecontrol signal.

In Step 1228, a substantial alignment between the target position andthe light source is detected within the FOV of the auxiliary imagingelement. In particular, the substantial alignment is a result ofadjusting the orientation of the camera device or a relative position ofthe camera device in the opposite direction to the displacement.

In Step 1229, in response to detecting the substantial alignment, animage of the scene is captured by the camera device. In one or moreembodiments, consecutive images are continuously captured and outputtedby the camera device at a regular repetition rate (i.e., camera framerate).

In Step 1230, a determination is made as to whether image capturing bythe camera device is to continue. If the determination is positive,i.e., the image capturing is to continue, the method returns to Step1222. If the is negative, i.e., the image capturing is not to continue,the method ends.

FIGS. 3.1, 3.2, 3.3, 3.4, 3.5, 4, 5, and 6 show various examples inaccordance with one or more embodiments of the invention. The examplesshown in FIGS. 3.1-6 may be, for example, based on one or morecomponents depicted in FIGS. 1.1 and 1.2 above and the method flowchartsdepicted in FIGS. 2.1 and 2.2 above. In one or more embodiments, one ormore of the modules and elements shown in FIGS. 3.1-6 may be omitted,repeated, and/or substituted. Accordingly, embodiments of the inventionshould not be considered limited to the specific arrangements of modulesshown in FIGS. 3.1-6.

FIG. 3.1 shows a camera mobile device handheld grip (800) as an exampleof the camera device holder (130) depicted in FIG. 1.1 above. Inaddition, a camera mobile device (201) (e.g., a smart phone having acamera lens (220)), mechanically held by the camera mobile devicehandheld grip (800), is an example of the camera device (110) depictedin FIG. 1.1 above. In particular, the camera lens (220) is associatedwith the imaging element A (111) to provide image focusing function. Inone or more embodiments of the invention, the camera mobile devicehandheld grip (800) is an electro-mechanical assembly that includes aholder (221), a tilting shaft (203), a tilting motor (213), a rotatingshaft (209), a rotating motor (219), and a handheld grip (222). Theholder (221) is configured to mechanically hold the camera mobile device(201) and mechanically couple to the tilting shaft (203). The handheldgrip (222) is configured to maintain, while being handheld by a viewer,mechanical stability of the camera mobile device handheld grip (800).Although not explicitly shown, the handheld grip (222) includes acommunication interface configured to communicate with the camera device(110) and/or the stabilization controller (120) depicted in FIG. 1.1above. For example, the communication interface may be based onBluetooth, NFC, USB, or other wireless/wired communication interfaces.In one or more embodiments, the rotating shaft (209) is rotatable arounda rotating axis (209-1) by the rotating motor (219) in response to acontrol signal received from the stabilization controller (120) via thecommunication interface. Similarly, the tilting shaft (203) is rotatableby the tilting motor (213) around a tilting axis (203-1) in response tothe control signal received from the stabilization controller (120) viathe communication interface. In response to tilting the holder (221)around the tilting axis (203-1) and/or rotating the holder (221),collectively with the tilting shaft (203) and tilting motor (213),around the rotating axis (209-1), the orientation of the camera lens(220) may be adjusted. Accordingly, the FOV (220-1) of the camera lens(220) is adjusted according to the orientation of the camera lens (220).Although the example shown in FIG. 3.1 is based on two motors associatedwith two mechanical shafts, other examples may be based on three motorsassociated with three mechanical shafts without departing from the scopeof the invention wherein the third motor may be an additional rotatingmotor, such as the additional rotating motor (331) with the additionalrotating axis (209-2) shown in FIG. 3.3 below. Specifically, FIG. 3.3shows a camera mobile device handheld grip (800) with three motors as anexample of the camera device holder (130) depicted in FIG. 1.1 above.

FIG. 3.2 shows an example of stabilizing the camera mobile devicehandheld grip (800) depicted in FIG. 3.1 above. For example, theorientation of the camera mobile device (201) is stabilized when thecamera mobile device handheld grip (800) is changed from the handheldposition A (321) to the handheld position B (322) or changed from thehandheld position B (322) to the handheld position A (321). As shown inFIG. 3.2, the handheld position A (321) corresponds to a verticalorientation (i.e., along the earth gravity direction) of the handheldgrip (222). In the handheld position A (321), the tilting motor (213)maintains the camera mobile device (201) pointing toward the earthhorizon (i.e., orthogonal to the earth gravity direction). In otherwords, the imaging plane of the camera mobile device (201) is orthogonalto the earth horizon.

The handheld position B (322) corresponds to a tilted orientation (i.e.,deviating from the earth gravity direction) of the handheld grip (222).For example, the tilting motion (323) of the handheld grip (222) isexerted by the user's hand. In the handheld position B (322), thetilting motor (213) maintains the camera mobile device (201) pointingtoward the earth horizon as in the handheld position A (321).

FIG. 3.4 shows a front view example of rigidly coupling the auxiliaryimaging element and the camera mobile device handheld grip (800)depicted in FIGS. 3.1-3.3 above. As shown in FIG. 3.4, the auxiliaryimaging element B (112) is integrated with the stabilization controller(120), which is rigidly coupled with the holder (221) of the cameramobile device handheld grip (800). Accordingly, the auxiliary imagingelement B (112) is rigidly coupled with the camera lens (220) andassociated imaging element A (111). FIG. 3.5 shows a rear view of theexample depicted in FIG. 3.4.

FIG. 4 shows an example of the light change pattern (124) of the lightsource (143) depicted in FIGS. 1.1 and 1.2 above. As shown in FIG. 4,the horizontal axis corresponds to time and the vertical axiscorresponds to light intensity. In particular, the light change pattern(124) is a pattern of light intensity alternating between a bright level(400 a) and a dark level (400 b) over time. For example, the brightlevel (400 a) of the light intensity sustains over a time period A (410)and may be recurring over time with certain repetition rate. While thelight intensity alternates between the bright level (400 a) and the darklevel (400 b) over time, a sequence of auxiliary images is captured bythe auxiliary imaging element periodically. For example, consecutiveimages in the sequence may be captured at a time point A (401 a), timepoint B (401 b), time point C (401 c), etc. that are separated from eachother by a time period B (420), time period C (430), etc. In particular,the time period A (410) encompasses at least one image capture timepoint, such as the time point B (401 b). Although the light changepattern (124) depicted in FIG. 4 is a pattern of light intensitychanges, the light change pattern (124) may also include color changesin other examples. In other words, the bright level (400 a) and darklevel (400 b) may be substituted or supplemented by different colors torepresent color changes.

FIG. 5 shows an example of the sequence of auxiliary images (126) of thescene (140) depicted in FIGS. 1.1 and 1.2 above. As shown in FIG. 5, thesequence of auxiliary images (126) includes the image A (126 a), image B(126 b), image C (126 c), etc. that are captured at the time point A(401 a), time point B (401 b), time point C (401 c), etc. depicted inFIG. 4 above. According to the example of the light change pattern (124)described in reference to FIG. 4 above, the light source (143) appearsas an alternating dark and bright spot at a location marked “a” in theimage A (126 a), image B (126 b), image C (126 c), etc. In contrast, thelight intensity remains substantially constant at another locationmarked “b” in the image A (126 a), image B (126 b), image C (126 c),etc. For example, the location marked “a” may be determined bysubtracting intensity values of corresponding pixels in the image A (126a) and image B (126 b) to generate the subtraction result (126 d).Similarly, the location marked “a” may be further determined bysubtracting intensity values of corresponding pixels in the image B (126b) and image C (126 c) to generate the subtraction result (126 d). Inthe subtraction result (126 d), black color indicates no difference andwhite color indicates a non-zero difference. Accordingly, the locationof the light source corresponds to the white spot in the subtractionresult (126 d).

Further as shown in FIG. 5, the center of each image is defined as thetarget position (127). Accordingly, the distance from the locationmarked “a” to the target position (127) corresponds to the displacement(125). The location marked “a”, the target position (127), and thedisplacement (125) shown in FIG. 5 are examples of the location A (126b), target position (127), and displacement (125), respectively,depicted in FIG. 1.2 above. In one or more embodiments, the locationmarked “a” varies between the image A (126 a), image B (126 b), image C(126 c), etc. The rate of change of the location marked “a” across imageA (126 a), image B (126 b), image C (126 c), etc. corresponds to thecontrolled movement parameter (125 b) depicted in FIG. 1.2 above. In oneor more embodiments, the displacement (125) and the controlled movementparameter (125 b) are measured based on image pixels.

FIG. 6 shows an example of stabilizing or object tracking of a cameradevice based on the sequence of auxiliary images (126) described inreference to FIG. 4 above. In an example scenario, the target positionis the center of the image. As shown in FIG. 6, the light source (143)is identified at a location in the left portion of the images (e.g.,image A (126 a)) in the sequence of auxiliary images (126). Inparticular, the light source (143) is held by both hands of a maleperson (i.e., object (142)). For example, the location of the lightsource (143) is identified based on the alternating dark and bright spotin the image A (126 a), image B (126 b), image C (126 c), etc. depictedin FIG. 5 above. In other words, the light source (143) corresponds tothe location marked “a” in the image A (126 a), image B (126 b), image C(126 c), etc. depicted in FIG. 5. Because the target position (i.e.,image center) is to the right of the light source location, the objectstabilization controller (120) is configured to orient the camera device(110) toward the left such that the male person (i.e., object (142))holding the light source (143) appears in the center of the image.Accordingly, the orientation of the camera device (110) is adjustedbased on the identified location “a” of the light source (143) such thatthe object (142) appears in the center of the image X (126 x), which iscaptured by the imaging element of the camera device.

Embodiments of the invention may be implemented on a computing system.Any combination of mobile, desktop, server, router, switch, embeddeddevice, or other types of hardware may be used. For example, as shown inFIG. 7.1, the computing system (700) may include one or more computerprocessors (702), non-persistent storage (704) (e.g., volatile memory,such as random access memory (RAM), cache memory), persistent storage(706) (e.g., a hard disk, an optical drive such as a compact disk (CD)drive or digital versatile disk (DVD) drive, a flash memory, etc.), acommunication interface (712) (e.g., Bluetooth interface, infraredinterface, network interface, optical interface, etc.), and numerousother elements and functionalities.

The computer processor(s) (702) may be an integrated circuit forprocessing instructions. For example, the computer processor(s) may beone or more cores or micro-cores of a processor. The computing system(700) may also include one or more input devices (710), such as atouchscreen, keyboard, mouse, microphone, touchpad, electronic pen, orany other type of input device.

The communication interface (712) may include an integrated circuit forconnecting the computing system (700) to a network (not shown) (e.g., alocal area network (LAN), a wide area network (WAN) such as theInternet, mobile network, or any other type of network) and/or toanother device, such as another computing device.

Further, the computing system (700) may include one or more outputdevices (708), such as a screen (e.g., a liquid crystal display (LCD), aplasma display, touchscreen, cathode ray tube (CRT) monitor, projector,or other display device), a printer, external storage, or any otheroutput device. One or more of the output devices may be the same ordifferent from the input device(s). The input and output device(s) maybe locally or remotely connected to the computer processor(s) (702),non-persistent storage (704), and persistent storage (706). Manydifferent types of computing systems exist, and the aforementioned inputand output device(s) may take other forms.

Software instructions in the form of computer readable program code toperform embodiments of the invention may be stored, in whole or in part,temporarily or permanently, on a non-transitory computer readable mediumsuch as a CD, DVD, storage device, a diskette, a tape, flash memory,physical memory, or any other computer readable storage medium.Specifically, the software instructions may correspond to computerreadable program code that, when executed by a processor(s), isconfigured to perform one or more embodiments of the invention.

The computing system (700) in FIG. 7.1 may be connected to or be a partof a network. For example, as shown in FIG. 7.2, the network (720) mayinclude multiple nodes (e.g., node X (722), node Y (724)). Each node maycorrespond to a computing system, such as the computing system shown inFIG. 7.1, or a group of nodes combined may correspond to the computingsystem shown in FIG. 7.1. By way of an example, embodiments of theinvention may be implemented on a node of a distributed system that isconnected to other nodes. By way of another example, embodiments of theinvention may be implemented on a distributed computing system havingmultiple nodes, where each portion of the invention may be located on adifferent node within the distributed computing system. Further, one ormore elements of the aforementioned computing system (700) may belocated at a remote location and connected to the other elements over anetwork.

Although not shown in FIG. 7.2, the node may correspond to a blade in aserver chassis that is connected to other nodes via a backplane. By wayof another example, the node may correspond to a server in a datacenter. By way of another example, the node may correspond to a computerprocessor or micro-core of a computer processor with shared memoryand/or resources.

The nodes (e.g., node X (722), node Y (724)) in the network (720) may beconfigured to provide services for a client device (726). For example,the nodes may be part of a cloud computing system. The nodes may includefunctionality to receive requests from the client device (726) andtransmit responses to the client device (726). The client device (726)may be a computing system, such as the computing system shown in FIG.7.1. Further, the client device (726) may include and/or perform all ora portion of one or more embodiments of the invention.

The computing system or group of computing systems described in FIGS.7.1 and 7.2 may include functionality to perform a variety of operationsdisclosed herein. For example, the computing system(s) may performcommunication between processes on the same or different system. Avariety of mechanisms, employing some form of active or passivecommunication, may facilitate the exchange of data between processes onthe same device. Examples representative of these inter-processcommunications include, but are not limited to, the implementation of afile, a signal, a socket, a message queue, a pipeline, a semaphore,shared memory, message passing, and a memory-mapped file.

The computing system in FIG. 7.1 may implement and/or be connected to adata repository. For example, one type of data repository is a database.A database is a collection of information configured for ease of dataretrieval, modification, reorganization, and deletion. DatabaseManagement System (DBMS) is a software application that provides aninterface for users to define, create, query, update, or administerdatabases.

The user, or software application, may submit a statement or query intothe DBMS. Then the DBMS interprets the statement. The statement may be aselect statement to request information, update statement, createstatement, delete statement, etc. Moreover, the statement may includeparameters that specify data, or data container (database, table,record, column, view, etc.), identifier(s), conditions (comparisonoperators), functions (e.g. join, full join, count, average, etc.), sort(e.g., ascending, descending), or others. The DBMS may execute thestatement. For example, the DBMS may access a memory buffer, a referenceor index a file for read, write, deletion, or any combination thereof,for responding to the statement. The DBMS may load the data frompersistent or non-persistent storage and perform computations to respondto the query. The DBMS may return the result(s) to the user or softwareapplication.

The above description of functions present only a few examples offunctions performed by the computing system of FIG. 7.1 and the nodesand/or client device in FIG. 7.2. Other functions may be performed usingone or more embodiments of the invention.

While the invention has been described with respect to a limited numberof embodiments, those skilled in the art, having benefit of thisdisclosure, will appreciate that other embodiments can be devised whichdo not depart from the scope of the invention as disclosed herein.Accordingly, the scope of the invention should be limited only by theattached claims.

1. A method to stabilize image capture for a camera device, comprising:capturing, at an auxiliary frame rate and using an auxiliary imagingelement rigidly coupled with the camera device, a plurality of auxiliaryimages of a light source attached to an object, wherein the auxiliaryframe rate exceeds a camera frame rate of the image capture; detectingeach of a plurality of locations of the light source in a correspondingone of the plurality of auxiliary images; analyzing, by a hardwareprocessor, the plurality of locations to generate a measure ofuncontrolled movement; and adjusting, based at least on the measure ofuncontrolled movement, the camera device during the image capture of theobject by the camera device at the camera frame rate.
 2. The method ofclaim 1, wherein the measure of uncontrolled movement represents adeviation from a predicted trajectory of the object across the pluralityof auxiliary images.
 3. The method of claim 2, wherein analyzing theplurality of locations comprises: determining the predicted trajectoryby at least applying a curve fitting algorithm to the plurality oflocations.
 4. The method of claim 3, further comprising: analyzing theplurality of locations to generate a measure of controlled movement,wherein the measure of controlled movement represents the predictedtrajectory of the object across the plurality of auxiliary images. 5.The method of claim 4, further comprising: mitigating, based on themeasure of controlled movement, a rolling shutter effect of the imagecapture.
 6. The method of claim 4, further comprising: generating, basedon the measure of uncontrolled movement and the measure of controlledmovement, a control signal, wherein the control signal adjusts, duringthe image capture, a field-of-view of the camera device to perform atleast one selected from a group consisting of image stabilization basedon the uncontrolled movement and object tracking based on the controlledmovement.
 7. The method of claim 6, further comprising: sending, duringthe image capture, the control signal to a camera device holder holdingthe camera device, wherein the control signal comprises at least oneselected from a group consisting of a rotating control signal, a tiltingcontrol signal, and an additional rotating control signal, whereinadjusting the field-of-view of the camera device comprises at least oneselected from a group consisting of: activating, using the rotatingcontrol signal, a rotating motor of the camera device holder;activating, using the tilting control signal, a tilting motor of thecamera device holder; and activating, using the additional rotatingcontrol signal, an additional rotating motor of the camera deviceholder.
 8. A stabilized camera device holder for stabilizing imagecapture, comprising: a computer processor and memory storinginstructions, when executed, causing the computer processor to: receive,from an auxiliary imaging element rigidly coupled with a camera device,a plurality of auxiliary images of a light source attached to an object,wherein the plurality of auxiliary images are captured by the auxiliaryimaging element at an auxiliary frame rate that exceeds a camera framerate of the image capture; detect each of a plurality of locations ofthe light source in a corresponding one of the plurality of auxiliaryimages; and analyze the plurality of locations to generate a measure ofuncontrolled movement; and at least one motor configured to: adjust,based at least on the measure of uncontrolled movement, the cameradevice during the image capture of the object by the camera device atthe camera frame rate.
 9. The stabilized camera device holder of claim8, the instructions, when executed, further causing the computerprocessor to: wherein the measure of uncontrolled movement represents adeviation from a predicted trajectory of the object across the pluralityof auxiliary images.
 10. The stabilized camera device holder of claim 9,wherein analyzing the plurality of locations comprises: determining thepredicted trajectory by at least applying a curve fitting algorithm tothe plurality of locations.
 11. The stabilized camera device holder ofclaim 10, the instructions, when executed, further causing the computerprocessor to: analyze the plurality of locations to generate a measureof controlled movement, wherein the measure of controlled movementrepresents the predicted trajectory of the object across the pluralityof auxiliary images.
 12. The stabilized camera device holder of claim11, the instructions, when executed, further causing the computerprocessor to: mitigate, based on the measure of controlled movement, arolling shutter effect of the image capture.
 13. The stabilized cameradevice holder of claim 11, the instructions, when executed, furthercausing the computer processor to: generate, based on the measure ofuncontrolled movement and the measure of controlled movement, a controlsignal, wherein the control signal causes the at least one motor toadjust, during the image capture, a field-of-view of the camera deviceto perform at least one selected from a group consisting of imagestabilization based on the uncontrolled movement and object trackingbased on the controlled movement.
 14. The stabilized camera deviceholder of claim 13, wherein the control signal comprises at least oneselected from a group consisting of a rotating control signal, a tiltingcontrol signal, and an additional rotating control signal, whereinadjusting the field-of-view of the camera device comprises at least oneselected from a group consisting of: activating, using the rotatingcontrol signal, a rotating motor of the camera device holder;activating, using the tilting control signal, a tilting motor of thecamera device holder; and activating, using the additional rotatingcontrol signal, an additional rotating motor of the camera deviceholder.
 15. The stabilized camera device holder of claim 8, wherein theauxiliary imaging element is integrated with at least one selected froma group consisting of the stabilized camera device holder and the cameradevice as a single device.
 16. The stabilized camera device holder ofclaim 8, wherein the auxiliary frame rate is between 6 times to 10 timesof the camera frame rate.
 17. A stabilization controller device forstabilizing image capture, comprising: a computer processor; and memorystoring instructions, when executed, causing the computer processor to:receive, from an auxiliary imaging element rigidly coupled with a cameradevice, a plurality of auxiliary images of a light source attached to anobject, wherein the plurality of auxiliary images are captured by theauxiliary imaging element at an auxiliary frame rate that exceeds acamera frame rate of the image capture; detect each of a plurality oflocations of the light source in a corresponding one of the plurality ofauxiliary images; analyze the plurality of locations to generate ameasure of uncontrolled movement; and adjust, based at least on themeasure of uncontrolled movement, the camera device during the imagecapture of the object by the camera device at the camera frame rate. 18.The stabilization controller device of claim 17, wherein the measure ofuncontrolled movement represents a deviation from a predicted trajectoryof the object across the plurality of auxiliary images.
 19. Thestabilization controller device of claim 18, wherein analyzing theplurality of locations comprises: determining the predicted trajectoryby at least applying a curve fitting algorithm to the plurality oflocations.
 20. The stabilization controller device of claim 17, theinstructions, when executed, further causing the computer processor to:analyze the plurality of locations to generate a measure of controlledmovement, wherein the measure of controlled movement represents thepredicted trajectory of the object across the plurality of auxiliaryimages.
 21. The stabilization controller device of claim 20, theinstructions, when executed, further causing the computer processor to:mitigate, based on the measure of controlled movement, a rolling shuttereffect of the image capture.
 22. The stabilization controller device ofclaim 20, the instructions, when executed, further causing the computerprocessor to: generate, based on the measure of uncontrolled movementand the measure of controlled movement, a control signal, wherein thecontrol signal causes a camera device holder of the camera device toadjust, during the image capture, a field-of-view of the camera deviceto perform at least one selected from a group consisting of imagestabilization based on the uncontrolled movement and object trackingbased on the controlled movement.
 23. The stabilization controllerdevice of claim 22, wherein the control signal comprises at least oneselected from a group consisting of a rotating control signal, a tiltingcontrol signal, and an additional rotating control signal, whereinadjusting the field-of-view of the camera device comprises at least oneselected from a group consisting of: activating, using the rotatingcontrol signal, a rotating motor of the camera device holder;activating, using the tilting control signal, a tilting motor of thecamera device holder; and activating, using the additional rotatingcontrol signal, an additional rotating motor of the camera deviceholder.
 24. The stabilization controller device of claim 17, wherein theauxiliary imaging element is integrated with at least one selected froma group consisting of the stabilization controller device and the cameradevice as a single device.
 25. The stabilization controller device ofclaim 17, wherein the auxiliary frame rate is between 6 times to 10times of the camera frame rate.
 26. A non-transitory computer readablemedium storing instructions for stabilizing image capture for a cameradevice, the instructions, when executed by a computer processor,comprising functionality for: capturing, at an auxiliary frame rate andusing an auxiliary imaging element rigidly coupled with the cameradevice, a plurality of auxiliary images of a light source attached to anobject, wherein the auxiliary frame rate exceeds a camera frame rate ofthe image capture; detecting each of a plurality of locations of thelight source in a corresponding one of the plurality of auxiliaryimages; analyzing the plurality of locations to generate a measure ofuncontrolled movement; and adjusting, based at least on the measure ofuncontrolled movement, the camera device during the image capture of theobject by the camera device at the camera frame rate.
 27. Thenon-transitory computer readable medium of claim 26, wherein the measureof uncontrolled movement represents a deviation from a predictedtrajectory of the object across the plurality of auxiliary images. 28.The non-transitory computer readable medium of claim 27, whereinanalyzing the plurality of locations comprises: determining thepredicted trajectory by at least applying a curve fitting algorithm tothe plurality of locations.
 29. The non-transitory computer readablemedium of claim 26, the instructions, when executed by the computerprocessor, further comprising functionality for: analyzing the pluralityof locations to generate a measure of controlled movement, wherein themeasure of controlled movement represents the predicted trajectory ofthe object across the plurality of auxiliary images.
 30. Thenon-transitory computer readable medium of claim 29, the instructions,when executed by the computer processor, further comprisingfunctionality for: mitigating, based on the measure of controlledmovement, a rolling shutter effect of the image capture.
 31. Thenon-transitory computer readable medium of claim 29, the instructions,when executed by the computer processor, further comprisingfunctionality for: generating, based on the measure of uncontrolledmovement and the measure of controlled movement, a control signal,wherein the control signal adjusts, during the image capture, afield-of-view of the camera device to perform at least one selected froma group consisting of image stabilization based on the uncontrolledmovement and object tracking based on the controlled movement.